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IDENTIFIERS/ 
ABSTRACT 

. , Vocational-technical education has neglected the area 

of policy research, not because Pederiar support preempts the 
policy-making role^ but' for a dombination of reasons: (1) adverse 
reaction to a vocational-technical program is not welcomed by its 
authors and benefactors, (2) vocational-technical educators are not 
typically researcherysr (3) the need for study of a pj^oblem is . 
precluded by the need for action, (4) researchers wifhin^and outside 
the field are not trusted to understand the problem^iii-and (5) 
educators in vocatipnal-technical education are teluctantf, to 
sacriMce any of tlje iiidependence presently' enjoyed by State and 
Ipeal leadership. vPolidy research is necessary, hpwever. T^e image ofi 
vocational-technicil edtjcation is s^ffering because th« evidence ^ 
supporting it^s worth has, not been collected* organized, and analyzed, 
studies which have been Conducted (l^auf mail and Schaefer* Haltet M. 
Arnold, Arthur n.'ie^) d6 not attempt to measure in an a6alytic sense 
what policy prodttdes the best results. This is the time to involve 
social and behavidral scientists in Vocational-technical education 
evaluation, to C09lvince |)olitical authorities that the data do 
support the conjWtures,; and to be an accountable member of the 
educational systeiiir\in terms of funding and policy, (^G) 



•POLIC^-}iESEARCH. AND NATIONAL AND STATE ' - ' 
"EVALUATION . STUDIES OF VOGATLONAL-TECHNIdAL EDUCATION , - . 

by ■ . .... 

* ■ . CARL J, SCHAEFER*'' , / ' / 

The topic of^poMcy research has surfaced, at least in 
the discipl-ine of, sociology, with vigor ^nd forceful debate 
indicative of those. stout and hearty souls that' make up this . 
disciplihe. James Coleman (1^72) began stirring the pot for 
policy, research in the social sciences as early as 1972 ,- when, 
he published his paper advocating that the^ social sc/entist? ^ 
/unction as policy scientists. ' Although the vColeman paper / t 
brought quick reaction, some t.6 the cohtr^ry from his fellow • 
sociologists, the fact remains th^ voc^ational-technical . • 
education has not* indulged itself in policy research either , 
witli dr without the 'assistance of this discipline* ' The question, ; 
therefore, is not necessary why^but why not? Lest such . ■ ^ 
rhetoric appears confused, precisely stated, Should we ' \ 

embark on some straightforward policy research and if so, * 
what'mi^t be the pay off to the improvement of bur field*?" 

As near as I can determine, after ^having been exposed/to ^ > 

^ . . \ ' ' . . • ' . , 

vocational education in a number of states, there is indeed 

aXi objective of thqse- in charge to, prove that the acceptance 

* ♦ ■ 

of federal vocational funds does not impinge upon the ^ 

prerogative of a state to. set policy. And to prove this, we ' , . 
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literally boast that • vocational education policy is as. different' 

in as many cases as we have states. Such divergently comes 

quickly to the fore whe^ one attends our national meetings 

knd hekrs time after ti/ne state leadership rising to depict ^ 

in. glowing terms how prdblems wei^ solVed in their. state in 

their unique manner ^nd with.^t5ut the weakest of appeal to - > 

others, reflecting, "tryfit you' 11. like it'" .. And, to show 

the complete independence^ on the part of fellow cohorts, ' ' . 

each returns home determined, not to try it . It is.no wonder, 

then,>Jhat one^^om^how gets the feeling that all that ' interests 

us i^n generation after generation of vocational educators is . 

the rediscovery of the wheel.. I submit our historical 

'forefathers, the P.ross6rfi ,• Wrights and Aliens, literally knew 

what we know today ..|Ld in some cases were practicing it much 

l^ette.i?*- . ■ ^ . 

In ilefense of those who do not "try it*' ds the fact that 

.too Ofteni cub position papers do^ not carry the hard facts to o . * 

support oUr conjectures. At one time I thought 'the establish- 

<' , . . * ' 

^ment of the State Research Coordinating Units would provide 

such hard datai but I am now* convinced that in the- main, they^ 

do not have..=the analytic capability for s.uch an Aindertaking* ' 

* *■ . ' • ■ ^ - , 

instead, we find in the Semiannual Report on^ Research Coordinating 

Unit Activities (1972)/ such bland undertaking^ as. publishing 

newsletters, providing microfiche readers, conducting career . 

development programs > special paper preparation and dissemination 

an^i the like* Rarely does one fin4 reported studies ^ of • a^ 

/ - -V 

follbw-^up nature, evaluative* undertakings and more basic type 



research endeavors.* And even ratfv does, one get to feel that 

*we ^have enlisted in our efforts -the research expertise of 

th0se.- related disciplines such as sociology, econoniics and 

psychology. I need not remifid y6u that. the Research .^Coordinating 

Units account for seyeral millions of dollars of the yearly 

federal ' vocational funds, • . » • ^ 

♦ ^The much talked about report entitled^ Work ' in Amex-'ica 

(1972) and the more 'recent ar^ticle appearing in the July, 1974' 
f * * •» • • . ^ . ' ' ' ^ * « 

issue of Manpower^ (1974) entitled "Vocational -Education: * 

. . . ' * ' ' . . 

.Performance and Potential" blafantly attaqks vocational^ ^ 

education. as being ineffective'^ ill conceived, and^in general,. 

. * . ■ ' •» 

'a poor investment. This latter article (Vocational Education: 

Performance and Potential) purpof^ts to have reviewed the 

' ' • * . . . < . . ' ^ ' 

findings^ of our research "Undertakings and answers such questions 

as : ' ' . . 

Are- fjraduates of* high school vocational' ' ^* 

progi ams who go straight to jobs, better • i . 

• off in . the' quality of<;jobs, earnings,. 

unemployment ^rate^ ,• 6r job satisfaction,, 
" than cojriparat>le non-vocational graduates-*. 
* ^ ^ /who go to work immediately? Is the ^ . 

nature- of the curriculum a causal or . • ^ . . 

decisive factor in any difference between 

these grouPiS? dther important issues i 

include the comparative drop out rates 
. ' ' of ^students in .various high school 

curricula and the effect of vocational 

education on the pursuit of higher 

education* , . . 

The conclusion j:^eached. was that vocatxona^l educa^ioh is 
not a very viable alternative' in . t^rms of. the issues raised* 
Yet the review of our re^arch^ is shaded t>y such ^ caveats .as: 



At the outset, some words of caution 
. are in 'ordgr /about the, studies .reviewed / . 

•. ' in- this article^ They are. beset with.. ' . 
* .• conceptiona^l problems , • methodological: . / 

• pitfalls, and statistical limitations,,^ * 
They often are not comparable. with«9ne ^ 
another, and few of them'were done recently . . » ^ 
^ 'fenough to measure the improvements in . 
high 'school/ vocational; education planned /. 
V • V by the 1968 amendments for the Vocational 
^Education Act. Proceeding from dj.fferent 
. values and assumptions, the analysts ',hAve 
not'e.ven agreed on the objectiy.es or ' 
' / ' outcomes to be tested; ' ^ ' ' 

• * . ' . • ' ■ ' •* 

' Having been* engaged i aL^g with Jacob Kaufman, in several 
• : . ( • . ' ' ^ 

studies of an evaluatfve nature of vocational-technical edu*^. 

cation I can sympathize with some frustra^tions alluded to by 

those who undertake to review our* field *^s limited and meager 

attempts to pro\/'e once and for all th^ worth of vocational 

education. I caij also attest to the. lack of full review of 

the Schaefer .and Kaufman.,^filtciings in the two previously , 

mentioned publioAtions. Our data, in* at least two studies ^ 

;*Kaufman, Schaefe^r, et al, 1967j *and Schaefer* and.. Kaufman V ^ 

1971), show that vocational education has a "pay off" and ^ . 

has a potential whigh is only now being reali^ed-'by the total 

educational ^profession and that is' using the "process of 

vx>cationd:l education" fo.r all youth* a$ so ably defined by 

former Commissioner Sidney P* Marland^ Jr»,*'in his" career,, 

^ . . . 

education concept • . . , , - 

It is easy to be "fouled out" when one id cfuoted'out of 
context o^ misquoted* altogether* Our "role" study represents 
the large'^st single sample of high! school . grAduates.'*ever 
intervicrWedi some. 5^181 ^ from vocational^ general, and 
academic. curricwlums correlated with some 2 ,626 on the job 



supervisors and o^e^ GOO^employ^rs "rating, instruments.- 
■Questionnaires were received from 1,600 teachers and to' 
check the data obtained from th^ interviews ♦ another 3,?42 
respondents .completed and returned , mailed questionnaires* 
Moreover', much of^ the analysis was controlled across the . 
sample by IQ scores, thus making it a quasi experimental- type, 
un^dertaking.' Our state-wide study '-.(fjlassachusetts) was not as 

-Vj^terralve but was based ox\ a questionnaire sample of 1,7^6 

:\ ■ 

stratified on vocational^ general, and academic" curipiculums 
• • 'fr 

In aclditibn, it included a study of '228. private vocational-i 
technical schools and 820 industrial employers. 
*[ Corvskquently , and especially . in the **role»' study, we * ^ 
were not 'extrapolating our data frorfi smalL,numb.ers and\the; 
findings, although needing updating, do supply some of the 
hard data-^nee(^ed even today. But like most evaluative 
studies, Kaufman *s and my work does not attack the problem^ 
of policy decisipVimaking. I suppose- we naively thought that^ 
once having accumulated hard data, some /policy "application 
would be evident. And I am afraid liiuch, fhe .same will happen 
from Arthur Lee*s (1972) national effort. It. has just been 
announced that\ the National Academy of Science ( Educational 
Researcher, 1974 X is in the process, of evaluating the overall 
research* and development effort of vocational education. It 
will' be interesting to see what their review will have to say 
about our policy research effort.^ 

• Other studies of an evaluative type which* have a bearing 
on the problem being discussed are Arnold*s*(1969) study .of 



Pennsylvania and some of the work of Somers (19 71) , S.trong 
•(197Q),' Stromsdorfer (1969), and Teh-wei Huiet%l (1969^,' as 
well as the National Planning Association (1972) study Policy 

Issues and Analytical Problems in Evaluating- Vocational • 

.'■''*** 

' Education * " 

Yet I would hasten, to poin^ out that none of these> at^e 
of the policy research type of which Coleman* speaks. They 
do not attempt to measure in an analytic sense what policy 
produces the^Jjest results. In fact, Kaufntan* and I studied * ' . 
very little of the policies found, except to get a "gut 
feeling" that the decision maimers for vocational educat-ion, 
whether at- the local or state levels, were too. low in the 
hierarchy to have much influence at the top. . And the only • 
success we may have had in this argument is the fact; that in 
Massachusetts the State Director fs tij,tle went frorti Chief of 
a Bureau to Associate Commissioner. 

This . being the case*, Coleman position that policy 
research is a most neglected area needing study is highly 
valid.. For example,* the fie3.d of vocational-technical 
education had been attacked time and time 'again for its 
r^esources distribution^ especially' the federal dollars^ach 
sti^te receives. Yet, we know that these federal dollars pi?oduc 
• varied results from state to state*: Some states support 
almost all of their salaried personnel in their division of 
vocational education; out of federal^dollars , while others use 
the same dollars fo^ almost entirely local program support . 
The pointx hei?e is that resource distribution policy mayj in 




one cAsV, produce mbre^in termV of desirable outcomes than 

in anoth'eV. But we -really do not know. Pplicy research, • " 

/ to use Coleman's term; can be carried on in terms of path 

designs to trace the effect the federal funds has on the 

intended recipient . A long organizational path may negate 
' ' ' ', • ♦ ■ 

♦impact of this policy on its ultimate recipients. In other 

words, too many federal do?.lars invested in state diyision 

* personnel may retard' rather than advance p'rograms. Coleman 

is enough of a'realist to. see the many problems involved in 

pure policy research which attefapts to measure policy inputs 

• / , ■ • 

and relate them to policy . outputs . One way of doing this. 

(besides experimental design) he suggests is through what he 

palls "social audit". More, specifically, Coleman states: 

.In a social audit, r^sourc^:^;l3t^ 
' initiated by policy are tmced W^ - 
the^ point at which tHey em dx^blirsed 
to the point at .vjhich the$\^are^ " 
. \ ' " ^experienced by the ultimate^'^wt ended ; ^ 

I 'recipient ofi those. resources It is ^ / . 

then those resources as experienced 
that are related to^ the outcomes in - y 
the research, rathe;r than the i?e sources y 
as disbursed^ For there are. two / 
possible causes for the inef fectivehess 
/of resources: the resources as^ '\ 
• * experienced may be ineffective in ^ . \ ^ 

bringing about any change.; or "tihe 
resources as disbursed' never reach the 
ultimate intended recipient and are 
instead lost somewliere in the path 
between point and initial disbursement . 
and th^ point of experience by the 
ultimate recipient. .In research that • * 

cloes not tr^ce J:he resources along thxs 
" • path, it is impossible to distinguish 

these two causes of .ineffectiveness and ^ - ^ 
the as^sumption is ordinarily m^de that 
resources as. experienced 'are the same ds 
resources as disbursed. (p* 18) ♦ - 



Wruile John Evans iCl97H) in a recent article in the 
Educational Researcher indicates some progress in evalu..-*-ive 
research endeavors and thei;? implication for policy making, 
.it remains, at least from my vantage point, to see anything on 
the horizon dealing specifically with vocational-technical educa- 
tion policy research, ."'in fact, unless our leadership rises above 



their present statute and, overcomes our image of "business 

as usual", I see very little chaace oil maintaining vocational- 

technical education as a discrete force in the educational 

■ .:■ . ' . ■ ■ ■ . • 

mosaic. 

Let*s face it: . . . 

i 1.. Once vocaticmal-technioal programs' are put into"' 

effect, the political force pf- their authorship arid the pr<?ssure 
from their constituency for continued funding makes It' almost , 
impossible to acknowledge publicly that they are m/t as 
effective as we would like. 'v • j- ■ 

2. ^ Vocational-technical educators are doer's and not 
experimenters. We believe actions speak louder than words" 
and we simply do not have the patience /to involvfe ^ourselves 

to.any great.exte^t in research and inquiry, whether it be 

ft 

for policy making or ^ven saving dollars ^ for that matter. 

3. Our leadership (state and local) is politically 
astute *to the point of making for bad education. Their' mode 

of policy decision making isl based on the principle — if there. 
' is enough' interest in some problem to support a majpr s^Udy, 

then the interest will' be so 'great t^at'^no one will be 
. willing to wait for,' the conclusion of the research before 



proceeding with the: program* On the other hand., if there is 
not a broad concern o\^er the problem, then there will not' be 
enough interest to fund the research anyway.. Bither way th>r^ 
'need not be any experimentation on the problem. 

1+.. We do not trust our. own researchers, let alone those 
^f^^pm other disciplines. We fear social and behavioral scientists- 
because they,. do not understand the probleVns of our, ongoing 
vocational programs and we are nSt willing to 'embrace them ■ 
Xong enough so they will get indfeplh .ir.tiights; into our 
.problems. f , . ' 

^ 5. We have steadfastly refused to use the much talked 
about but little understood repearch tools of Program Pl<£nning 
and Budgeting (PPBS) and Management by Objectives (MBO) 

because they taKe away the V authoritarian" .mode! of state and j 

■ ' ■ ' ■ ' ■ ' ' \ 

local leadership. ' - ■ . \ 

•« 

It precisely because of these reasons why we do not' v 
do policy research/ Yet there is no time more important than, 
now for: 1) involvsiment of the social and behavioral 
scient-ats in the evaluation of • vocational-technical education ; / 
2 ) the need • for coi\viricing the political powers that be , that 
we haVe the data to support our conjectures and; 3) that we 
are accountable for our part in t^ie ^educational mosaic, both 
in terms of funding and policy. The j>ay off of policy research 
coUld well produce the leverage that vocational-technical 
educatich so badly needs. 

J .* ' ' iO ' ' 
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